{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "U9i2Dsh-ziXr"
      },
      "source": [
        "# Eager Execution Tutorial: Basics\n",
        "\n",
        "This notebook introduces the basics of using TensorFlow's eager execution capabilities. It covers concepts such as:\n",
        "\n",
        "* Importing required packages\n",
        "* Enabling eager execution\n",
        "* Creating and using TensorFlow Tensors and Variables\n",
        "* Using TensorFlow interactively\n",
        "* Using GPUs with eager execution enabled\n",
        "\n",
        "This notebook does *not* cover modeling topics, such as gradients."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "z1JcS5iBXMRO"
      },
      "source": [
        "# Step 1: Import Eager\n",
        "\n",
        "The key imports for eager execution are the following:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "RlIWhyeLoYnG"
      },
      "outputs": [],
      "source": [
        "# Import TensorFlow.\n",
        "import tensorflow as tf\n",
        "\n",
        "# Import TensorFlow eager execution support (subject to future changes).\n",
        "import tensorflow.contrib.eager as tfe"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "H9UySOPLXdaw"
      },
      "source": [
        "# Step 2: Enable eager execution\n",
        "\n",
        "All future TensorFlow calls will execute the\n",
        "underlying TensorFlow ops immediately:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "WPTUfGq6kJ5w"
      },
      "outputs": [],
      "source": [
        "tfe.enable_eager_execution()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "twBfWd5xyu_d"
      },
      "source": [
        "# Step 3: Interactively Use TensorFlow!\n",
        "\n",
        "Now you can call TensorFlow functions and get results, immediately! No more `tf.Sessions`!\n",
        "\n",
        "TensorFlow will automatically wrap native Python types for you with operator overloading for TensorFlow Tensors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "ngUe237Wt48W"
      },
      "outputs": [],
      "source": [
        "print(tf.add(1, 2))\n",
        "print(tf.add([1, 2], [3, 4]))\n",
        "print(tf.square(5))\n",
        "print(tf.reduce_sum([1, 2, 3]))\n",
        "print(tf.encode_base64(\"hello world\"))\n",
        "print(\"\")\n",
        "\n",
        "x = tf.constant(2)\n",
        "y = tf.constant(3)\n",
        "print(x * y + 1)\n",
        "\n",
        "# Most TensorFlow ops are directly usable with eager execution, giving\n",
        "# results immediately.\n",
        "print(tf.contrib.signal.hamming_window(x * y + 1))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "IDY4WsYRhP81"
      },
      "source": [
        "Numpy arrays are supported, too:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "lCUWzso6mbqR"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "ones = np.ones([3, 3])\n",
        "\n",
        "print(\"numpy 3x3 matrix of 1s:\")\n",
        "print(ones)\n",
        "print(\"\")\n",
        "\n",
        "print(\"Multiplied by 42:\")\n",
        "print(tf.multiply(ones, 42))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "PBNP8yTRfu_X"
      },
      "source": [
        "# Step 4: Define and Print TensorFlow Variables\n",
        "\n",
        "To define TensorFlow variables, use the `get_variable()` function as follows:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "3Twf_Rw-gQFM"
      },
      "outputs": [],
      "source": [
        "x = tf.get_variable(name=\"x\", shape=[], dtype=tf.float32, initializer=tf.zeros_initializer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "45G7094TxsMb"
      },
      "source": [
        "## Printing TensorFlow Variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "UJBJeZ5XxuwA"
      },
      "outputs": [],
      "source": [
        "# This does NOT print the Variable's actual value:\n",
        "print(\"Printing a TensorFlow Variable:\")\n",
        "print(x)\n",
        "print(\"\")\n",
        "\n",
        "# A TensorFlow variable represents a reference to a tensor.\n",
        "# The `read_value()` method provides access to the current value of the\n",
        "# variable. Tensorflow Variables are automatically initialized according to the\n",
        "# semantics defined in tf.get_variable().\n",
        "print(\"Printing a TensorFlow Variable's value using .read_value():\")\n",
        "print(x.read_value())\n",
        "print(\"\")\n",
        "\n",
        "print(\"Printing a TensorFlow Variable's value using .read_value().numpy():\")\n",
        "print(x.read_value().numpy())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "2njjWHcTpBEn"
      },
      "source": [
        "## Changing a TensorFlow Variable's value\n",
        "\n",
        "To change a TensorFlow Variable's value, use its `.assign()` or `.assign_add()` method:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "v3wr6Erbo_hB"
      },
      "outputs": [],
      "source": [
        "x.assign(42)\n",
        "print(x.read_value())\n",
        "\n",
        "x.assign_add(3)\n",
        "print(x.read_value())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "uhtynjHVpTB5"
      },
      "source": [
        "## Use a Variable just like any other Tensor"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "7PbktdnHoehR"
      },
      "outputs": [],
      "source": [
        "print(x + 3)\n",
        "\n",
        "# This code will broadcast the value across the list of numbers:\n",
        "print(x * [1, 2, 4])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "GVChqwlwy1SI"
      },
      "source": [
        "# Step 5: Debug Errors with Instant Feedback\n",
        "\n",
        "TensorFlow's eager execution helps you identify and debug runtime issues through interactive exploration of code snippets.\n",
        "\n",
        "Below, we'll define a length-4 vector, and attempt two `tf.slice()` operations,\n",
        "one being legal and the other being illegal, leading to a runtime error that is\n",
        "raised immediately."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "23ap04N0v4k0"
      },
      "outputs": [],
      "source": [
        "vector = tf.constant([10.0, 20.0, 30.0, 40.0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "FCUMsIYxxRRa"
      },
      "outputs": [],
      "source": [
        "# Works, because the values of `begin` and `size` (the 2nd and 3rd input\n",
        "# arguments) are within the bound of `vector`.\n",
        "print(tf.slice(vector, [1], [3]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "T8me2oCNxpFp"
      },
      "outputs": [],
      "source": [
        "# The following does NOT work, because the value of `size` (the 3rd\n",
        "# argument) causes the indices to go out of the bounds of `vector`. The\n",
        "# error is raised immediately.\n",
        "try:\n",
        "  print(tf.slice(vector, [1], [4]))\n",
        "except tf.OpError as e:\n",
        "  print(\"Caught error: %s\" % e)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "irxJhAgar84v"
      },
      "source": [
        "# Step 6: Using the GPU\n",
        "\n",
        "You can place Tensors on the GPU by calling a Tensor's `.gpu()` method.\n",
        "\n",
        "The first operation executing on the GPU may be slow as TensorFlow initializes. Subsequent uses will be much faster."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "7J4N9baqaKCL"
      },
      "outputs": [],
      "source": [
        "# The example code from here on will work only if your notebook\n",
        "# is running on a machine with a functional CUDA GPU. The following\n",
        "# line checks that.\n",
        "is_gpu_available = tfe.num_gpus() \u003e 0\n",
        "\n",
        "# Create some Tensors\n",
        "SIZE = 1000\n",
        "cpu_tensor = tf.random_normal([SIZE, SIZE])\n",
        "\n",
        "if is_gpu_available:\n",
        "  gpu_tensor = cpu_tensor.gpu()\n",
        "else:\n",
        "  print(\"GPU not available.\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "4E-2n7VbzY1n"
      },
      "outputs": [],
      "source": [
        "# Time a CPU-based matrix multiplication\n",
        "\n",
        "print(\"Time to conduct matmul on CPU:\")\n",
        "%time tf.matmul(cpu_tensor, cpu_tensor)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "vbSFW-T5zhZF"
      },
      "outputs": [],
      "source": [
        "# Time GPU-based matrix multiplications.\n",
        "\n",
        "if is_gpu_available:\n",
        "  # First use of the GPU will be slow:\n",
        "  print(\"Time to conduct first matmul on GPU:\")\n",
        "  %time tf.matmul(gpu_tensor, gpu_tensor)\n",
        "  print()\n",
        "\n",
        "  # Subsequent uses are much faster:\n",
        "  print(\"Time to conduct second matmul on GPU:\")\n",
        "  %time tf.matmul(gpu_tensor, gpu_tensor)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "E5pIOe3Rz7iW"
      },
      "outputs": [],
      "source": [
        "# Second timing demo for GPUs, after it has been used once:\n",
        "\n",
        "cpu_tensor = tf.random_normal([SIZE, SIZE])\n",
        "print(\"Time to conduct CPU matmul:\")\n",
        "%time tf.matmul(cpu_tensor, cpu_tensor)\n",
        "print()\n",
        "\n",
        "if is_gpu_available:\n",
        "  gpu_tensor = cpu_tensor.gpu()\n",
        "  print(\"Time to conduct GPU matmul:\")\n",
        "  %time tf.matmul(gpu_tensor, gpu_tensor)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "default_view": {},
      "name": "Eager Execution Tutorial: Basics",
      "provenance": [
        {
          "file_id": "0B0kLcpwLFwKEVm9XNkFueGk4bTg",
          "timestamp": 1504118841551
        }
      ],
      "version": "0.3.2",
      "views": {}
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
